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Abstract 

I shall describe a general model-theoretic task to construct expan¬ 
sions of pseudofinite structures and discuss several examples of particular 
relevance to computational complexity. Then I will present one specific 
situation where finding a suitable expansion would imply that, assuming 
a one-way permutation exists, the computational class J\fV is not closed 
under complementation. 

Consider the following situation: M is a nonstandard model of true arith¬ 
metic (in the usual language of arithmetic 0,1, n is a nonstandard el¬ 
ement of M, L is a finite language and W £ M is its interpretation on the 
universe [n] = (1,... , n}; W can be identified with a subset of [n k ] for some 
UN. We shall denote the resulting structure A w] it is coded by an element of 

k 

M that is < 2 n . Without a loss of generality we shall assume that L contains 
constants 1, n, the ordering relation < interpreted as in M, and ternary relation 
symbols © and 0 for the graphs of addition and multiplication inherited from 
M. Because M is a model of true arithmetic Ayy is pseudofinite: it satisfies the 
theory of all finite A-structures. 

Paris and Dimitracopoulos m studied the problem of for how large m > n 
does the theory of the arithmetic structure on [n] determine the theory of the 
arithmetic structure on [to] and proved that it does not for m = 2 ra . They 
also pointed out various links between questions of this type and complexity 
theory problems around the collapse of the polynomial time hierarchy. Ajtai 
[T] showed (among other similar results) that if M is a countable nonstandard 
model of PA and L is finite then for any L-structure Ayy there are two sets 
U, U' C [n], both elements of M, such that M thinks that \U\ is odd and \U'\ is 
even while, as structures, (Ayy, U) = (A w, U') (the isomorphism is not in M, of 
course jjj. Krajicek and Pudlak m showed that for any nonstandard f<ntM 

1 We shall discuss another example with parity later. 
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one can construct M' D M containing a proof of contradiction in PA of length 
bounded by n* without adding any new elements to interval [0, n]. Mate pj3] 
considered the full second order structure on [n] (coded in M) and reformulated 
the statement that MV ^ coMV as a statement about non-preservation of the 
theory of the structure in an expansion coming from a model M' A M. 

The most interesting results of this kind (to this author) were obtained 
initially by Ajtai mm- In the first paper he established that parity of n bits 
cannot be computed by AC 0 circuits (proved independently by Furst, Saxe 
and Sipser 0) and he reports there that his first proof of the lower bound 
was by model theory of arithmetic although he eventually chose to present 
the result combinatorially. In the second paper he proved that propositional 
formulas PHP m formalizing the pigeonhole principle do not have polynomial 
size constant depth Frege proofs. That proof is by constructing a suitable model 
of arithmetic!!. We shall discuss these two examples in the next section. 

In this, mostly expository, note we are interested in the general question of 
how to construct expansions of Aw with particular properties. Before formu¬ 
lating this more specifically we will consider in the next section three examples. 
The examples go back to Ajtai mm and (essentially) Mate jT8| but two of them 
are not presented in the literature with enough details and are formulated with 
unnecessarily strong hypotheses. In the subsequent section we shall discuss a 
specific open problem whose solution would have interesting implications for 
computational complexity. 

The note is self-contained modulo a basic knowledge of logic and complexity 
theory. Notions not explained here can be likely found in [TZ] . 


1 Examples 

Parity example. It is well-known that the parity of a string of bits cannot be 
computed by AC 0 circuits, [E0. That is, 

(1) For any d > 1 and large enough m, any depth d , size < m d circuit with 
m inputs must compute erroneously the parity of some m-bit string. 

Computability by AC 0 circuits is equivalent to first-order definability in the 
presence of an extra structure (of a fixed signature, the extra structure depend¬ 
ing just on the size of the universe). In particular, (1) is equivalent to 

(2) For any finite language L and any formula <f>(X) in the language L(X), 
L augmented by a unary predicate X(x), for m large enough and any 
T-structure B with universe [to] it holds: 

2 The original manuscript was only about models of bounded arithmetic. After Ajtai learned 
that Paris and Wilkie m linked provability of PHP in bounded arithmetic with a conjecture 
of Cook and Reckhow [6] that formulas PHPm are hard for Frege systems, shown eventually 
false by Buss [5], he added a few hand-written pages showing how his result implies a lower 
bound for constant depth Frege systems. 
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• There is U C [to] such that the equivalence: 


(B, U) |= <&(U) iff | U | is odd 


fails. 

Using overspill in M, (2) is equivalent to the same statement for any L-structure 
on any nonstandard [n] in M (with U £ M and its parity defined in M). And 
that can be further formulated as follows. For u £ [n] denote U <u := {v £ 
U \ v < 11 } and U- u := {w £ U \ v < u}. 

(3) For any nonstandard n £ M, any finite language L , any L(X) formula 
<F(X) and any L-structure £ M with universe [n] it holds: 

There is U C [n], U £ M, and u £ U such that the following holds: 

(a) 

(A W ,U <U ) |= $(?/<“) iff (A W ,U^ U ) (= $([^“) . 

(b) For t £ U <u , 

(A W ,U<*) iff (A ff ,^‘)M(^‘)- 

Let y (x) be a new unary predicate and consider first-order theory Tf in the 
language L(X,Y), L(X) augmented by Y, with axioms: 

1. The least number principle axioms: 

3x a(x) —> (3x\!y < x a(x) A -<a(y)) 

for all formulas a in the language L(X, Y) (as we will evaluate formulas 
over a structure with universe [n] they are de facto bounded), 

2. 3x £ X, (xeY)^ $(X- X ), 

3. axiom ’l'(y), where T is the formula: 

y C X A min A £ Y A (Vy £ X, sucx(min X) = y —>• y £Y) A 

Vx, y £ X, sucx (sucx (x)) = y —> x £ Y = y GY 
where for x £ X, sucx(x) is min{;z £ X \ x < z}, if it exists. 

Claim 1: Statement (3) for a given language L, formula set U £ M and 
some u £ U is equivalent to the existence ofVQ [n] (V not necessarily in M) 
such that the expanded structure (A w,U,V) satisfies T® (U interprets X and 
V interprets Y). 

For the only if direction note that for U and u € U satisfying (3) a suitable V 
can be defined in M already: take for V the subset of U consisting of its elements 
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on odd positions. Axiom 1. holds because M satisfies it for all formulas, axiom 
3. holds by the definition of V and axiom 2. holds as it is witnessed by x := u. 

For the if direction assume that (A^, U , V) |= Tj*. Take for u the minimal 
x witnessing axiom 2.; it exists by the least number principle. Utilizing axiom 
3. we see that the pair U,u satisfies statement (3). 

PHP example. In this example we aim at a proof complexity lower bound. 
Given m > 2, consider a propositional formula PHP m formed using atoms py, 
i £ [m] and j £ [m — 1] that is the disjunction of the following formulas: 

• V, A, /V 

• V il5 « 2 VjOiU Ap ia j), 

• ViVj 15 y 2 (Piji A /''/•.< )• 

Having a falsifying assignment py := a,j £ {0,1} we could define the graph of 
an injective map from [m] to [m — 1]: 

{(bj) I a-ij = 1} 

which is impossible. So PHP m is a tautology. 

The depth of a formula (in DeMorgan language) is the maximal number of 
connected blocks of alike connectives on a path of subformulas of the formula 
(the depth of PHP m is thus 3). A Frege system F is a sound and implicationally 
complete finite collection of inference rules and axiom schemes. Its depth d 
subsystem Fd is allowed to use only formulas of depth < d. 

A path in a depth d formula of size (i.e. the number of symbols) s can be 
naturally determined by a d-tuple of numbers < s, giving d pointers to which 
subformula the path moves when alternating from one connective to another. 
This implies that such a formula can be coded by a d-ary function on [s] giving 
information about atoms (or constants) in which individual paths end. Hence 
an Fd proof of PHP n of size polynomial in n can be coded by a relation S on 
[n]. The qualification naturally used above means that, given d, one can define 
in ([n],i?, S) the satisfaction relation between formulas in the proof S and a 
truth assignment^!?. 

Using this, and overspill as before, the statement 

(4) For all d > 3 and all large enough m > 2 there is no Fd proof of PHP m 
of size < m d , 

is equivalent to the following statement about M and any nonstandard n £ M: 

(5) for all d > 3 and any relation S on [n], 5 £ M, S does not encode an Fd 
proof of PHP n . 

Let L be a finite language and let Zbea variable for a binary relation. Consider 
the theory T 2 : 

3 Details left out in this example can be found in m- 
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• Induction axioms for all L(.Z)-formulas as in T®, 

• -^PHP(Z) axiom: 

[\/xBy < n Z(x, y)\ A [Va; < x'\/y < n -<Z(x, y) V ~<Z(x', y)] A 
[Va :\/y < y' < n -> Z(x,y) V ~^Z(x,y')] . 

Claim 2: Assume that for any finite L and any L-structure Aw with universe 
[n] (any nonstandard n £ M) there exists an expansion (A\y,R) satisfying T 2 
with Z := R. Then statement (5) is true. 

To see this assume that (5) fails, i.e. some relation S on [n] encodes an 
FVproof of PHP n . Let (Aw, R) be a model of T 2 , with W containing S. Using 
the truth definition for depth < d formulas show that under the assignment 

Pij := 1 if R(i,j) and pij := 0 otherwise 

all propositional axioms and the formula -<PHP n are true. Hence, by induction, 
there has to be an inference whose all hypotheses are true while its concussion 
is not. But that is impossible. 

Note that Claim 2 is formulated as an implication and not equivalence as 
Claim 1; we shall return to this issue in the next section. 

TAUT example. Let TAUT C (0,1}* be the set of propositional tautologies 
in the DeMorgan language. A propositional proof system (abbreviated to PPS) 
in the sense of Cook and Reckhow [6] is a polynomial time binary relation P on 
(0,1}* such that 

• Vx,y P{x,y ) —* x G TAUT (soundness), 

• Va: G TAUT —> 3y P(x,y) (completeness). 

A PPS P is p-bounded if there exists k > 1 such that the By in the completeness 
can be bounded by |y| < \x\ k . A p-bounded PPS exists iS AfV = coMV , see [6]- 
The main task is therefore to establish for all PPSs a super-polynomial lengths- 
of-proofs lower bound. This may be far away at present but lower bounds 
for specific PPSs have interesting consequences as well (e.g. for independence 
results or for SAT algorithms). 

As before we shall consider strings of length polynomial in n (in particular, 
formulas) coded by relations on [n]. For a given PPS P and k > 1 there exists 
by Fagin’s theorenfl a formula 0(A, Y, Y’) such that BY'Q(X, Y, Y') defines on 
[n] the relation P(x,y) A |y| < \x\ k . 

Denote by SAT(Z , A) a first order formula defining the satisfaction relation 
between an assignment Z and a DNF formula X (we want to avoid coding of 
evaluations of general formulas in this discussion). Define a theory T 3 with 
axioms: 

4 Invoking just Fagin’s theorem here is not enough for various properties one needs often 
from 0. The formalization needs to be ’’natural” again. The claim below holds for arbitrary 
0 defining the relation, though. 
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• -<SAT(Z,X), 

• MX',Y,Y',Z' 9{X',Y,Y') -> SAT(Z',X'). 

Note that it is a nj theory, not first order as T-f and T 2 . 

Claim 3: Assume that for any finite L, any L-structure Aw with universe [n] 
and any standard k > 1 there is F C [n], F € M a DNF formula that is a 
tautology in M, such that (A WiF) has an expansion (A\ViF,R) satisfying 
with X := F and Z := R. Then P is not p-bounded. 

If P were p-bounded with exponent \x\ k then by overspill the formula F 
would have a P- proof in M of size polynomial in n, and Q(F, S, S') would hold 
for some relations S, S' £ M on [n] that we can put into W. Hence an expansion 
that is a model of X 3 for X := F is impossible. 


2 Discussion 

Claims 2 and 3 can be established as equivalence statements using the theory 
of propositional translations of n] theories fcf.|121 Chpt.9]). For that argument 
to work one does not need that M is a model of true arithmetic but only that 

(a) 2 n exists in M for some nonstandard t < n, 

(b) M satisfies bounded arithmetic theory R\ (which yields induction for E] 
formulas on Aw), 

(c) M is countable. 

Ajtai 0 01 formulated a general existence theorem for theories T going well 
beyond first order or n] theories. Such T can be not only second order or third 
order, etc., but it can be a finite set theory over Ur-elements [n] (and even more), 
and he allows not only expansions of Aw but expansions of end-extensions of 
Aw- The existence of such a model of T is characterized by the non-existence 
of a proof of contradiction in T that is - in a specific, rather technical, sense - 
definable over Aw- We refer the reader to Garlfk |9] who found a simpler and 
more conceptual proof of Ajtai’s theorem. The construction needs M satisfying 
(a)-(c) above and also 

(d) L is finite. 

In Claims 1 - 3 we stipulated that L is finite in order to avoid the discussion 
how it is coded. In these claims L can be, in fact, infinite as long as Aw is 
coded in M. But in 00U9] the hypothesis (d) is needed. 

The intended goal of Claims 1 - 3 is to offer a strategy how to prove a lower 
bound in complexity theory by constructing a suitable expansion of Aw- This 
has been done by Ajtai mm for the lower bounds explained in Claims 1 and 2. 
Ajtai mm works in a model of PA but the construction needs only assumptions 
(a) and (c) above and a variant of (b): 
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(b’) M is a model of the theory PV and of the weak pigeonhole principle for 
p-time functions (as represented by PV-terms), denoted WPHP(PV). 

See QH Sec.15.2] for how the WPHP(PV) is used. 

It is a challenge to construct a suitable model of X 3 in the situation of 
Claim 3 for strong PPS P (or even for all PPSs). In an attempt to meet the 
challenge particular models M' (extending a cut in M) were constructed in 
jl6l f!4] and [13] (two different constructions). They were constructed under the 
assumptions that a one-way permutation exists. These models M' satisfy the 
following conditions: 

1. 2 2 " exists in M' for some standard e > 0 (this is stronger that (a)), 

2. M' is a model of the true VIlJ theory in the language of PV (this does not 
imply either (b) or (b’)). In particular, all PPSs are sound in M'. 

3. L is infinite and coded in M' but Aw is not coded in M', 

4. there is a DNF formula F C [n], F £ M (the original model of true 
arithmetic) that has the form: 

V Ti( x ,V l ) 

i£[n\ 

with each ipi(x,y‘) having the form 

M x >y l ) -t Vi(y l ) 

and such that: 

(i) F is a tautology in both M and M', 

(ii) x, y 1 , ..., y n are mutually disjoint tuples of variables, 

(iii) there exists assignments A € W and B l £ W for all i £ [n] such that 
ipi(A,B l ) fails, i.e. SAT((A, B l ), -npi), and thus S AT ((A, B l ) , ipi) A 
SAT(B\^i)i), hold in M'. 

We now derive more properties of M' using the assumption that a PPS P is 
p-bounded and sufficiently strong. We shall use the specific form of the formula 
F used in HUES; we will formulate its properties explicitly but the interested 
reader is assumed to learn the definition of the formula in Hina and check 
that it has the stated properties. 

Claim A.Assume P is p-bounded and sufficiently strong. Then, in M', the 
following facts hold: 

5 We need that P contains resolution, P-proofs of true sentences can be constructed by a 
p-time algorithm (hence its soundness is true in M'), and that P simulates modus ponens 
and substitutions of constants with only a polynomial increase in the proof length. 
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(iv) There is a P-proof o/Vig[„] M' coded by a relation on [n\, 

(v) for each i G [n] there are P-proofs of ( ipi{x,z ) A il>i(x,z')) —*■ z = z’ and 
of ~«pi(A, y l ) in M' coded by relations on [n], 

(vi) formula Aie[n] not P-refutable in M'. 

Statement (iv) follows from the p-boundedness of P and (i) above. The first 
part of statement (v) follows again from the p-boundedness of P and the fact 
that formulas (ipi(x,z) A ifi(x,z')) -» z = z' are tautologies in M. It follows 
that ’ipi{A,B l ) —> ip i {A,y' 1 ) and thus ’ipil y A 1 y l ) (using that the true sentences 
ipi(A,B l ) have P-proofs) and ^r]i(B l ) have P-proofs in M' too. Statement (vi) 
is valid because of the specific formulas ifi have the property that it holds in M 
that 

Mx3y = {(y) 1 ,(y) n ) /\ tp(x, ( y)i) . 

iS[n] 

This implies that in M for no a and k € N there is a P-proof of -i A ;g[n] V’A 0 ) 2/*) 
of size < n k , and by p-boundedness of P these facts have P-proofs in M (and 
hence in M') and P is sound in M'. 

Ideally we would like to bring the existence of M' with properties 1.-4. (with 
(iv)-(vi) added) to a contradiction. That would imply that the hypothesis of the 
existence of a one-way permutation used in mm contradicts the hypothesis 
of p-boundedness of P used in Claim 4, entailing a conditional lower bound for 
(possibly all) P. In light of Claim 4 it seems natural to try to extend model M' 
by adding a satisfying assignment for /\ i if(A,y' 1 ). 

The existence of a satisfying assignment for /\ i i/j(A 1 y' 1 ) would follow if M' 
would satisfy a collection scheme for E] formulas on Aiy. But that is unlikely as 
the argument of Cook and Thapen [7] implies that the true VIIJ theory does not 
prove the scheme (unless factoring is not hard), and the same argument implies 
that one cannot argue just using the VIIj 1 theory of M' that it has an extension 
where the collection holds (we would also need that it adds no new elements 
into [n]). The construction of M' in |T5] depends just on the VIlJ theory of M 
and hence it is unlikely that it can be used again with the ground model being 
M'. 

The use of the property that we add no new elements into [n] in Claims 1-3 
was solely to preserve the first order theory of Aw (in the preceding paragraph 
it would also imply that (iii) above remains true in an extension of M'). There 
is an alternative for arranging that. In the forcing from hue] one naturally 
adds many new elements into [n] (and, in fact, does not include all of [n\ from 
M into M') but the first order theory of Aw is nevertheless often preserved. 
The set-up of the method presupposes some approximate counting available 
in the ground model (here M 7 ) and the model is assumed to be Hi-saturated. 
The former can be arranged as one modify the constructions so that the so 
called dual WPHP for p-time functions, dWPHP(PV), is true in M' too and 
that yields some approximate counting by Jerabek mm As for the latter 
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condition: some saturation property of M' could be arranged (cf. (Taj) if one 
could modify the forcing construction of M' so that it is defined by a compact 
family (in the sense of [16] ) of random variables. That would be possible if one 
could establish a hard-core lemma for the computation model underlying the 
construction. That, together with the fact (see 3. above) that A w is not coded 
in M', seem to be the main technical obstacles to apply Claim 3 to a general 
PPS. 

Ackowledgements: I thank Neil Thapen (Prague) for comments on a draft of 
this paper. 
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